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Paper  Abstract 


Since  their  introduction,  robots  and  the  technology  that  drives  them  have  continued  to 
evolve  rapidly.  That  technology  is  now  on  the  verge  of  becoming  fully  autonomous  and  lethal. 
Many  are  fearful  and  outright  oppose  the  development  of  Lethal  Autonomous  Systems  (LAS)  or 
what  some  call  “killer  robots.”  However,  when  we  look  back  twenty  years  from  now,  and  LAS 
have  replaced  man  on  the  battlefield,  many  of  those  same  people  and  organizations  will  look  back 
and  wonder  why,  despite  their  efforts,  they  could  not  stop  the  employment  of  such  systems.  Despite 
calls  for  a  ban,  lethal  autonomous  systems  (LAS)  will  see  widespread  future  use  in  the  military  for 
three  main  reasons  -  the  militarization  of  law  enforcement  and  border  protection  has  provided  a 
ripe  testing  ground,  commercially  designed,  dual  use  systems  are  easily  converted  to  lethal 
systems,  and  the  long-standing  illusion  that  we  can  somehow  control  the  context  of  a  weapon’s 
introduction  and  eventual  employment.  They  are  not  here  yet,  and  though  they  may  be  a 
technological  inevitability,  there  is  still  time  to  develop  a  framework  to  govern  LAS  employment 
in  the  military  and  all  other  sectors.  With  a  US-led  consensus,  the  international  community  can 
solidify  norms  and  standards  for  the  most  ethical,  moral,  and  legal  employment  of  LAS  in  wartime 
and  peace. 


iii 


Introduction  and  Thesis 


Since  their  introduction,  robots  and  the  technology  that  drives  them  have  continued  to 
evolve  rapidly.  That  technology  is  now  on  the  verge  of  becoming  fully  autonomous  and  lethal 
There  is  much  debate  over  the  ethics  of  the  use  of  lethal  autonomous  robots  in  the  military.  There 
has  been  strong  opposition  and  serious  ethical  concerns  voiced  at  the  highest  international  levels 
over  the  use  of  such  technology. 1  However,  when  we  look  back  twenty  years  from  now,  and 
these  “killer  robots”  have  replaced  man  on  the  battlefield,  many  of  those  same  people  and 
organizations  will  look  back  and  wonder  why,  despite  their  efforts,  they  could  not  stop  the 
employment  of  such  systems.  While  the  debate  continues  about  the  use  of  killer  robots  in  the 
military,  the  applicability  and  continued  use  of  the  same  kinds  of  technology  in  other  sectors  is 
paving  the  way  for  their  inevitable  lethal  use  in  the  military.  Despite  calls  for  a  ban,  lethal 
autonomous  systems  (LAS)  will  see  widespread  future  use  in  the  military  for  three  main  reasons 
-the  militarization  of  law  enforcement  and  border  protection  has  provided  a  ripe  testing  ground, 
commercially  designed,  dual  use  systems  are  easily  converted  to  lethal  systems,  and  the  long¬ 
standing  illusion  that  we  can  somehow  control  the  context  of  a  weapon’s  introduction  and 
eventual  employment. 

Unless  we  act  now  to  develop  an  ethical  framework  for  how  we  plan  to  use  these 
technologies  in  all  sectors,  it  will  be  too  late  to  govern  their  eventual  lethal  use  in  the  military 
successfully.  Before  we  tackle  these  main  arguments,  it  is  important  to  understand  the 
technological  progression  of  robotics  and  the  ethical  concerns  with  their  continued  evolution 
toward  lethal  autonomy. 


1  Among  these  opponents  are  groups  such  as  Human  Rights  Watch’s  “Campaign  to  Stop  Killer  Robots,’’  Eton  Musk 
led  and  $1B  funded  non-profit  “OpenAI,’’  and  the  Future  of  life  Institute  which  has  signatures  of  over  14,000 
researchers  demanding  the  ban  of  LAS 
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From  a  Remote  Controlled  Boat  to  the  Terminator 


“At  the  time  I  really  thought  it  would  abolish  war,  because  of  its  unlimited  destructiveness  and  exclusion 

of  the  personal  element  of  combat”  -  Nikola  Tesla 

In  2002,  the  Central  Intelligence  Agency  used  a  remotely  piloted  aircraft,  or  drone,  to 
engage  and  kill  six  suspected  terrorists  in  Yemen  from  a  control  station  thousands  of  miles  away. 
This  is  widely  held  as  the  first  such  event  of  its  kind.2  This  tactical  strike  was  executed  less  than 
one  year  after  testing  the  capability  with  retrofitted  Hellfire  missiles  on  an  RQ- 1  Predator,  a 
platform  originally  built  solely  for  reconnaissance.3  In  the  civilian  world,  this  type  of  tactical 
application  of  remotely  piloted  vehicles  was  unheard  of.  However,  in  2016  police  in  Dallas 
utilized  a  ground-based  robotic  platform  to  remotely  deliver  an  explosive  charge  and  bring  an 
end  to  a  standoff  with  a  suspect  who  killed  five  Dallas  policemen.4  Both  these  incidents  showed 
that  after  many  years  of  developing  the  technology,  particularly  in  the  military,  it  was  finally 
proving  to  be  tactically  beneficial.  However,  it  has  been  a  long  road  to  acceptance  and  for  the 
technology  to  get  to  this  point. 

In  1898,  Nikola  Tesla  demonstrated  a  radio-controlled  boat  capable  of  carrying  a 
warhead  to  the  American  government.  At  the  time,  the  military  did  not  see  the  value  in  his 
invention,  and  without  funding,  the  idea  faded  and  was  almost  forgotten.5  During  the  two  World 
Wars,  more  efforts  were  put  forward  that  produced  such  technologies  as  a  remotely  piloted 
aircraft  used  as  a  rudimentary  cruise  missile,  a  drone  controlled  from  an  aircraft  twenty  miles 

2  Christian  Enemark,  Armed  Drones  and  the  Ethics  of  War  (New  York:  Routledge,  2014),  19. 

3  Ibid.,  16. 

4Kevin  Sullivan,  Tom  Jackman  and  Brian  Fung,  “Dallas  police  used  a  robot  to  kill.  What  does  that  mean  for  the 
future  of  police  robots?,”  Washington  Post,  July  21,  2016,  17  November  17,  2016, 

https://www.washingtonpost.com/national/dallas-police-used-a-robot-to-kill-what-does-that-mean-for-the-future-of- 

police-robots/2016/07/20/32eel  14e-4a84-l  Ie6-bdb9-701687974517  storv.html 

5  Annin  Krishnan,  Killer  Robots:  Legality  and  Ethicality  of Autonomous  Weapons  (Surrey:  Ashgate,2009),  15. 
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away  and  “flying  bombs”  from  B-17s  and  B-24s.  The  German  and  Japanese  efforts  produced 
such  platforms  as  the  V- 1  jet  propelled  aircraft  which  carried  a  2000  pound  warhead  150  miles 
with  some  rudimentary  guidance,  the  Goliath  remote-controlled  tank,  and  Japanese  tanks  which 
could  be  remote  controlled  and  automatically  reloaded.6  Unfortunately,  all  suffered  the  same  fate 
as  Tesla’s  remote  controlled  boat  and  never  became  militarily  practical  during  their  time  due  to 
the  high  costs,  low  accuracy,  and  technology  gaps  that  could  not  be  closed. 

The  next  major  leap  towards  autonomous  technology  came  during  Vietnam  through  two 
major  initiatives.  On  the  border  of  South  Vietnam,  a  network  of  air  delivered  seismic  and 
acoustic  sensors  were  laid  to  prevent  infiltration  of  personnel  and  supplies  from  the  north.  This 
anti- infiltration  strategy,  called  the  McNamara  line,  relied  almost  exclusively  on  sensor  activated 
anti-personnel  mines  and  virtually  no  ground  forces.7  Second,  the  emergence  of  small  weapons 
and  “fire  and  forget”  systems  were  introduced.  The  effectiveness  of  these  weapons  was  still 
relatively  low,  and  while  they  were  nowhere  near  a  fully  autonomous  system,  they  laid  the 
foundation  of  two  key  components  critical  to  the  future  development  of  autonomous  systems. 

The  decision  to  fire  was  no  longer  reliant  on  visual  recognition  of  a  target  from  a  human  and  the 
weapons  themselves  were  automated  to  follow  cues  from  sensors  to  find  and  attack  their  targets.8 

With  the  promise  and  potential  shown,  these  technologies  continued  to  improve  steadily. 
Automated  self-defense  systems  such  as  Aegis,  Phalanx,  and  Patriot  were  bom  from  such 
technologies  in  the  70’s  and  80’s.  After  a  dip  in  funding  post-Cold  War,  the  wars  in  Iraq  and 
Afghanistan  have  brought  the  use  of  drones  and  unmanned  systems  to  the  forefront  of  combat 
once  again.  Since  2001,  unmanned  systems  have  poured  into  both  theaters  for  a  myriad  of 


6  Ibid.,  16-19. 

7  Ibid.,  20. 

8  Ibid.,  21 
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mission  sets.  Explosive  ordnance  disposal,  intelligence,  surveillance  and  reconnaissance,  surface 
and  subsurface  maritime  patrolling,  and  special  operations  are  just  a  few  of  the  missions  in  which 
these  platforms  have  been  utilized.  The  total  number  of  systems  in  unmanned  aircraft  platforms 
alone  jumped  from  less  than  100  in  the  year  2000  to  almost  11,000  in  20 13. 9  In  the  168-page 
FY2013-2038  Unmanned  Systems  Integrated  Roadmap,  the  word  autonomous  is  mentioned  155 
times. 10  Currently,  Department  of  Defense  (DoD)  Directive  3000.09  mandates  that  appropriate 
levels  of  judgment  over  the  use  of  force  should  be  designed  into  autonomous  and  semi- 
autonomous  systems.* 11  However,  Pentagon  officials  have  left  the  door  open  to  the  possibility  of 
taking  humans  “out  of  the  loop”  should  our  competitors  decide  to  do  so  first. 12  With  Moore’s 
Law  in  effect,  developments  in  fields  such  as  nanotechnology,  artificial  intelligence,  robotics, 
sensors,  and  information  technology  are  combining  to  make  a  fully  autonomous  lethal  system 
more  and  more  of  a  possibility.  As  this  day  draws  closer,  calls  to  examine  the  ethics  of  use  are 
growing  louder  as  well. 

The  Ethics  of  Autonomous  Killing 

As  the  technology  involved  continues  to  develop  in  both  the  military  and  civilian 
markets  rapidly,  the  technical  feasibility  of  lethal  autonomous  systems  (LAS)  is  becoming  less  of 
a  question  of  “if’  and  more  of  a  question  of  “when.”  The  real  debate  among  ethicists,  military 
professionals,  scholars,  and  industry  is  now  more  about  “should”  we  allow  LAS  on  the 


9  United  States  Department  of  Defense,  Unmanned  Systems  Integrated  Roadmap:  FY  2013-2038, 14-S-0553 
(Washington  DC,  2013),  5,  http  .'//archive,  defense.  sov/intbs/DQD-USRM-20 1 3  .pdf 

10  Ibid. 

1 1  United  States  Department  of  Defense,  DoDD  3000.09,  Autonomy  in  Weapons  Systems,  Washington,  DC: 
Undersecretary  of  Defense,  November  21,  2012,  2,  http://www.dtic.mil/whs/directives/corres/pdf/300009p.pdf 

12  Frank  Sauer  "Stopping  'Killer  Robots’:  Why  Now  is  the  Time  to  Ban  Autonomous  Weapons  Systems."  Anns 
Control  Today  46,  no.  8  (October,  2016),  13,  accessed  November  18,  2016, 
http://search.proquest.com/docview/1829055999?accountid=322 
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battlefield  when  they  become  technically  feasible  in  the  near  future.  Those  in  favor  for  their  use 


feel  that  LAS  could  protect  our  soldiers  in  harm’s  way  and  as  the  technology  advances  past 
human  capabilities,  could  be  more  humane  for  noncombatants.  Those  opposed  to  the 
employment  of  LAS,  generally  do  so  for  two  ethical  reasons.  First,  that  removing  the  risk  to 
soldiers  altogether  changes  the  nature  of  warfare.  Second,  and  most  importantly,  allowing  a 
machine  to  make  the  decision  to  kill  a  human  is  a  violation  of  our  most  basic  human  dignity. 

The  Ethical  Case  for  Killer  Robots 

Some  feel  that  in  a  purely  utilitarian  sense  if  LAS  could  prove  to  be  both  more  cost 
effective  and  more  capable  than  humans  in  the  performance  of  certain  mission  sets  they  should 
be  employed.  Only  the  results  should  matter. 13  A  growing  intolerance  for  casualties  on  the 
battlefield  combined  with  the  rapid  growth  of  technology  is  driving  major  trends  in  warfare  to 
make  systems  ‘faster,  reach  further,  and  operate  longer”  particularly  in  areas  that  are  difficult  for 
their  human  counterparts  to  operate.14  The  Pentagon  defines  these  as  environments  which  are 
dull,  dirty  or  dangerous.15  In  many  of  these  cases,  such  as  a  nuclear,  chemical  or  biological 
environment,  an  unmanned  system  is  the  preferred  alternative. 16  Additionally,  in  these  cases,  the 
enemy  has  a  vote  and  the  moral  decision  not  to  employ  the  technology  by  one,  does  not  prohibit 
adversaries  from  doing  so.  In  such  cases,  human  soldiers  might  be  put  in  a  decidedly 
disadvantageous  position  and  unable  to  react  to  the  speed  of  an  enemy  LAS. 


13  Kirshnan,  Killer  Robots,  121. 

14  M.  Shane  Riza,  Killing  Without  Heart:  Limits  on  Robotic  Warfare  in  an  Age  of  Persistent  Conflict  (Washington, 
D.C.,  Potomac  Books,  2013),  39. 

15  DoD,  Unmanned  Systems  Integrated  Roadmap,  20. 

16  Ibid. 
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Others  feel  that  in  addition  to  protecting  military  personnel,  LAS  could  make  war  more 
humane  for  noncombatants  as  well  While  there  are  still  technological  hurdles  to  overcome,  LAS 
could  eventually  be  programmed  free  of  human  emotions  such  as  anger,  spite,  fear,  or  panic  and 
could  lead  to  situations  where  less  lethal  force  was  taken  which  would  reduce  unnecessary  deaths 
in  war. 17  As  sensors  continue  to  become  more  precise,  autonomous  systems  will  be  able  to 
assume  risks  on  behalf  of  noncombatants  that  humans  constrained  by  the  fog  of  war  simply 
cannot  safely  take.  As  Georgia  Tech’s  Ron  Arkin  puts  it  “one  day  killer  robots  could  be  so 
precise  that  it  might  become  inhumane  not  to  use  them.”18 

Finally,  like  many  other  weapons  of  war,  LAS  are  merely  a  tool  of  warfare  implemented 
by  a  state.  The  mere  fact  that  a  state  possesses  LAS  doesn’t  change  that  state’s  traditional  Jus  ad 
Bellum  moral  calculus  for  going  to  war,  in  fact  an  army  of  robots  could  possibly  serve  in  a 
deterrence  capacity  to  prevent  war  from  happening  in  the  first  place. 19  If  war,  as  Clausewitz  tells 
us,  is  a  continuation  of  politics  through  other  means,  LAS  would  only  be  utilized  in  the  context 
of  achieving  political  goals  once  the  moral  decision  to  go  to  war  had  been  made.20  Their  use 
would  depend  on  the  ultimate  aims  of  a  nation  at  war.  If  LAS  could  be  shown  to  be  better  and 
more  efficient  at  achieving  those  goals,  there  may  not  be  any  moral  objections  strong  enough  to 
make  their  deployment  “morally  impermissible.”21 


17  UN  warns  stop  killer  robots  now.  Flare.  July  15,  2013,  53. 

http://www.lexisnexis.coin/lnacui2api/api/versionl/getDocCui?oc=00240&hnsd=f&hgn=t&lni=58T8-2MHl-JBlX- 

W09T&hns=t&perma=true&hv=t&hl=t&csi=384359&secondRedirectIndicator=true 

18  Eyder  Peralta,  “Weighing  the  Good  and  the  Bad  of  Autonomous  Killer  Robots  in  Battle”.  NPR,  accessed 
November  18,  2016,  http://search.proquest.com/docview/17867169197accountkL322 

19  Christopher  J.  Spinelli,  The  Rise  of  Robots:  The  Military’s  Use  of  Autonomous  Lethal  Force  (Research  Report, 

Air  War  College,  2015).  5 

20  Carl  von  Clausewitz.  On  War.  Eds.  andtrans.  Michael  Howard  and  Peter  Paret,  Princeton:  Princeton  University 
Press,  1976 

21  Duncan  Purves,  Ryan  Jenkins,  and  Bradley  J.  Strawser.  "Autonomous  Machines,  Moral  Judgment,  and  Acting  for 
the  Right  Reasons."  Ethical  Theory  and  Moral  Practice  18,  no.  4  (August,  2015):  851-72, 
http://search.proquest.com/docview/17005247577accountkL322 
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The  Ethical  Case  against  Killer  Robots 


The  first  major  objection  to  LAS  centers  on  the  reduction  in  risk  to  the  military 
professional  due  to  the  increasing  distance  between  the  human  warfighter  and  the  battlefield. 

LAS  detractors  believe  that  risk  is  an  essential  part  of  war  and  without  it,  participants  are 
conducting  something  that  is  very  un- warlike. 22  For  hundreds  of  years,  however,  military 
commanders  have  attempted  to  reduce  the  unnecessary  risks  to  their  personnel  while  attempting 
to  make  the  battlefield  more  lethal  for  their  opponents.  In  the  future,  LAS  may  make  it  possible 
to  eliminate  risk  to  soldiers  (  altogether).  In  the  past,  the  willingness  to  die  for  a  cause  has  been 
seen  as  part  of  the  moral  calculus  for  going  to  war.  Part  of  the  nobility  and  honor  of  a 
warfighter’s  cause  has  been  the  thought  that  they  were  willing  to  sacrifice  themselves  for  a  better 
life  for  their  people  in  the  future.  In  turn,  the  civilian  population  has  accepted  that  the  killing 
their  warfighters  must  undertake  in  combat  is  morally  just.23  What  then  becomes  the  moral  value 
of  war  when  one  side  is  pursuing  their  goals  with  impunity  from  risk  for  all  their  soldiers  in  the 
field?  Even  today,  the  pilot  flying  an  unmanned  aircraft  from  thousands  of  miles  away  might  be 
more  aptly  compared  to  a  professional  assassin  or  executioner  than  a  military  professional24 
This  absence  of  risk  from  the  warfighter  does  not  mean  that  risk  disappears  from  the  battlefield. 

In  the  case  of  LAS,  the  risk  eliminated  from  human  soldiers  is  sometimes  transferred  to 
the  noncombatants  on  the  battlefield.  Should  LAS  become  the  future  of  modem  warfare,  it  will 
be  the  first  time  in  human  history  where  those  responsible  for  fighting  the  war  would  be  at  less 
risk  than  noncombatants  on  both  sides.  In  this  case,  the  rules  of  war  we  use  today  may  begin  to 
crumble.25  Even  with  a  more  precise  and  efficient  LAS,  mistakes  will  be  made.  Hie  notion  that 


22  Enemark,  Armed  Drones,  76 

23  Ibid.,  81 

24  Ibid.,  94. 

25  Riza,  Killing  Without  Heart,  45. 
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all  civilian  casualties  would  be  eliminated  is  unrealistic,  but  at  what  cost  are  they  weighed 
against?  There  would  be  no  more  comparisons  of  combatant  versus  noncombatant  deaths  when 
weighing  a  cost  of  war  to  a  society.  Others  fear  that  lowering  the  risk  to  a  country’s  own  blood 
and  treasure  would  significantly  reduce  the  barriers  to  going  to  war. 

Once  the  deeds  and  actions  necessary  in  warfare  that  generate  risk  to  the  human  soldier 
have  been  separated  from  human  moral  decision  making,  the  inhibitions  to  go  to  war  will 
become  lower  as  well.26  As  the  technology  proliferates  and  becomes  more  available  and  cheaper, 
the  monetary  cost  of  entry  into  war  will  follow  suit.  For  those  states  that  may  not  be  able  to 
afford  to  organize,  train  and  equip  large  militaries,  an  army  of  cheap  LAS  may  seem  appealing. 
Those  nations  looking  for  an  asymmetric  advantage  may  not  share  the  same  moral  compass  as 
those  who  choose  to  withhold  such  systems  from  the  battlefield  for  moral  reasons.  This  approach 
will  surely  be  appealing  for  non- state  actors.  This  leaves  wars  to  be  fought  by  belligerents  who  in 
principle  do  not  kill  for  what  many  modem  societies  would  deem  as  acceptable  reasons.27  Once 
the  decision  to  kill  has  been  transferred  entirely  to  a  LAS,  it  stirs  an  even  more  fundamental 
objection. 

The  second,  and  most  vocal  argument  against  LAS  is  that  transferring  the  decision  to  kill 
a  human  to  a  machine  undermines  basic  human  dignity  and  the  mere  existence  of  this  capability 
is  an  unprecedented  threat  to  humanity.28  Many  argue  that  a  human  should  always  be  “in  the 
loop”  when  the  decision  to  take  another  human  life  is  being  made.  This  is  often  referred  to  as 
meaningful  human  control  and  goes  much  deeper  than  the  Pentagon  standard  of  “appropriate 
level  of  human  judgment.”29  Some  legal  scholars  have  made  the  argument  that  when  the  human 


26  Kirshnan,  Killer  Robots,  128 

27  Purves,  Ryan  Jenkins,  and  Bradley  J.  Strawser,  “Autonomous  Machines,”  865 

28  Kirshnan,  Killer  Robots,  4 

29  Sauer,  "Stopping 'Killer  Robots” 
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is  removed  from  the  loop,  it  is  a  violation  of  human  rights  law  and  impacts  on  the  concept  of 
human  dignity.30  Additionally,  the  potential  lack  of  responsibility  from  a  LAS  is  a  violation  of 
humanitarian  rights  law  as  well31  Should  a  LAS  commit  a  violation  of  the  Law  of  Armed 
Conflict  or  specific  rules  of  engagement,  who  would  be  responsible?  Is  it  the  military 
commander,  the  software  developer,  the  manufacturer,  or  some  other  entity? 

No  matter  how  sophisticated  and  advanced  artificial  intelligence  ultimately  becomes, 
there  are  worries  that  it  will  never  reach  a  human’s  ability  for  judgment  and  ability  to  read  the 
finer  context  of  a  situation.32  Regardless  of  how  effective  a  machine  might  become  in  making 
life  and  death  decisions,  the  technology  will  never  relieve  humans  from  the  ethical  impact  of 
those  decisions  in  both  peace  and  war.33  If  we  transfer  the  decision  to  kill  to  a  LAS,  the  system 
may  not  truly  understand  the  value  of  human  rights  and  we  may  cross  a  moral  threshold  from 
which  we  cannot  retreat.  34  Without  this  understanding  and  reflection,  war  is  in  danger  of 
becoming  nothing  but  mechanical  slaughter.35 

While  both  sides  of  the  debate  about  the  ethics  of  LAS  use  in  warfare  have  valid  points,  a 
solution  is  nowhere  in  sight.  Most  of  the  debate  has  focused  solely  on  the  military  application  of 
the  technology,  but  has  done  little  to  address  the  rapidly  maturing  technology  itself  and  the 
applications  outside  the  military.  As  the  discussion  continues  at  the  highest  levels  of  government 
and  international  organizations,  one  thing  is  clear  -  those  developing  the  technology  are  not 
waiting  for  answers  and  are  finding  other  markets  to  tap  into  until  the  military  markets  are  ready 


30  Denise  Garcia.  “Killer  robots :  Why  the  US  should  lead  the  ban”.  Global  Policy  6,  no.l  (January,  2015):  61 

31  Ibid. 

32  Alex  Brown.  “One  congressman’s  crusade  to  save  the  world  from  killer  robots.  The  Atlantic,  July  17,  2014, 
accessed  November  17,  2016,  https://www.theatlantic.com/politics/archive/2014/07/one-congressmans-crusade-to- 
save-the-world-from-killer-robots/442179/ 

33  “UN  warns  stop  killer  robots  now” 

34  Peralta,  “Weighing  the  Good  and  the  Bad” 

35  “UN  warns  stop  killer  robots  now” 
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to  cross  the  line.  When  the  question  is  finally  settled  about  the  appropriateness  of  use  for  the 
military  we  may  look  back  and  see  that  the  technological  genie  is  already  out  of  the  bottle  and 
the  decision  was  made  too  late  to  put  it  back  in.  Hie  international  community  did  not  have  an 
opportunity  to  stop  the  development  of  the  atomic  bomb  prior  to  its  use  as  they  do  here  with 
LAS.  They  were  only  able  to  limit  the  weapon’s  use  after  its  destmctive  potential  had  already 
been  demonstrated  twice  -  at  a  great  cost  to  civilian  lives.  Despite  the  fears,  there  are  many 
markets  outside  of  the  military  providing  ample  motives  to  continue  developing  LAS  technology 
which  will  provide  them  a  path  to  being.  Two  of  the  markets  that  will  serve  as  a  testing  ground 
and  pave  the  way  for  LAS  in  the  military  are  domestic  law  enforcement  and  border  protection. 

The  Militarization  of  Domestic  Law  Enforcement 

The  ongoing  militarization  of  fields  such  as  law  enforcement  and  border  protection  have 
created  a  demand  signal  for  very  similar  capabilities  that  are  offered  in  non-lethal  LAS  role.  In 
these  non-lethal  or  “less  than  lethal”  roles,  the  autonomous  systems  utilize  the  same  technologies 
of  the  more  lethal  variants  to  locate,  target,  and  fire  some  type  of  weapon  system.  However,  in 
“less  than  lethal”  systems,  something  such  as  a  Taser,  rubber  pellets  or  tear  gas  are  loaded  in  the 
place  of  bullets.  Even  these  systems,  if  used  improperly,  could  produce  lethal  results. 

Multinational  criminal  activity,  drug  and  human  trafficking,  and  terrorism  are  just  a  few 
mission  sets  that  have  transcended  international  boarders  and  created  broad  threats  to  national 
security.  This  phenomenon  has  tended  to  blur  the  lines  between  the  Department  of  Defense  and 
all  levels  of  traditional  law  enforcement  agencies.  It  has  also  transferred  some  of  the  risks 
associated  with  the  missions  from  the  military  to  law  enforcement.  Law  enforcement  personnel, 
like  those  in  the  military,  have  signed  a  moral  contract  to  sacrifice  their  life,  if  necessary,  to 
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protect  and  serve.  Much  like  soldiers  who  feel  out  of  place  risking  their  lives  overseas  on 
humanitarian  missions,  more  violent  cross-border  crime  and  terrorism  is  changing  the  moral 
contract  for  law  enforcement  officers  and  making  some  feel  that  “this  isn’t  what  I  signed  up 
for.”36  Due  to  these  threats,  domestic  policing  is  becoming  a  more  lethal  profession.  In  2016, 
ambush  style  killings  were  up  167%  for  police  officers  in  the  United  States,  and  the  total  of  64 
officers  shot  and  killed  was  a  five-year  high.37  In  the  case  of  terrorism,  officers  are  facing 
enemies  bound  by  no  rules  and  who  commonly  target  first  responders  to  increase  the  fear  and 
confusion  post- attack.  Under  the  banner  of  these  challenges,  Congress  has  authorized  the 
Department  of  Defense  to  enhance  domestic  law  enforcement  through  direct  or  material 
support.38  The  introduction  of  this  initiative,  known  as  the  1033  Program,  has  opened  the  door 
for  the  militarization  of  law  enforcement  and  a  similar  reliance  on  military  style  technologies  and 
equipment  for  mission  accomplishment. 

In  1989,  the  National  Defense  Authorization  Act  requested  the  DoD  take  the  single  lead 
between  federal  and  local  agencies  in  the  integration  of  the  command  and  control  and 
surveillance  networks  for  illegal  dmg  production  and  trafficking.  This  was  followed  with 
authorities  to  directly  transfer  excess  equipment  to  federal  and  state  agencies  that  were  suitable 
for  use  in  those  type  activities.39  In  1997,  this  authority  was  extended  to  counter-terrorism 
activities  and  redubbed  the  1033  Program.40  In  Section  1052  of  the  2016  NDAA,  the  1033 


36  Kirshnan,  Killer  Robots,  122. 

37  Susan  Miller,  "Officer  down:  Police  deaths  hit  a  5-year  high."  USA  Today,  29  December,  2016,  accessed  March 
12,  2017,  https://www.usatodav.com/storv/news/nation/2016/12/29/officer-down-police-deaths-hit-5-vear- 

high/95984998/. 

38  Daniel  H.  Else  “The  “1033  Program,”  Department  of  Defense 

Support  to  Law  Enforcement,”  (Congressional  Research  Service  #  R43791),  Washington,  DC:  Congressional 
Research  Service,  2014,  https ://fas .org/s gp/crs/natsec/R43701.pdf 

39  Ibid. 

40  Ibid.,  2 
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Program  was  expanded  directly  to  the  Department  of  Homeland  Security.41  Since  its  inception, 
over  11,000  agencies  have  registered  in  the  program  and  8,000  of  them  have  received  equipment 
at  no  cost  as  the  equipment  is  characterized  as  “on  loan.”42  Hie  transfer  program  includes  items 
as  benign  as  office  furniture  up  to  lethal  systems  such  as  weapons,  armored  vehicles,  robotics, 
and  manned  and  unmanned  aircraft.  The  value  of  the  equipment  transfers  since  1990  exceeds  $5 
billion  dollars,  $1.4  billion  of  which  was  tactical  equipment.43  Of  particular  note,  this  number 
includes  the  transfer  of  987  reconnaissance  and  bomb  disposal  robots  since  2003. 44  Over  3,900 
items  have  gone  to  DHS  which  includes  numerous  robots  and  drones. 

With  the  missions  and  equipment  sets  continuing  to  blur,  so  are  the  risks  to  personnel. 
Senior  officers  and  federal  agents  alike  see  this  technology  as  a  way  to  reduce  risk  to  their 
personnel  in  the  field.  In  My  2016,  a  gunman  killed  five  Dallas  police  officers  and  proceeded  to 
barricade  himself  in  a  building,  continuing  to  fire  at  officers  from  inside.  Seeing  no  other  option, 
Dallas  police  utilized  their  bomb  disposal  robot  to  deliver  an  explosive  charge  and  detonated  it  to 
remotely  eliminate  the  threat.  Police  Chief  David  Brown  stated,  “other  options  would  have 
exposed  our  officers  to  grave  danger.”45  He  also  added,  “I  would  use  any  tool  necessary  to  save 
our  officers’  lives.”46  Similar  feeling  were  expressed  by  a  senior  border  patrol  agent  in  response 


41  Nafeez  Ahmed,  "Pentagon  Allowed  to  Supply  Military  Gear  to  Homeland  Security  Department  for  "War  on 
Immigrants'"'  Truth  out.  14  May  2016,  accessed  November  17,  2016,  http://www.truth-out.org/news/item/35998- 
pentagon-allo  wed-to-supply-milit  ary -gear-to-homeland-security -depart  ment-for- war-on-immigrants. 

42  Else,  “The  1033  Program,”  1 

43  Shawn  Musgrave,Tom  Meagherand  Gabriel  Dance,  "The  Pentagon  Finally  Details  its  Weapons-for-Cops 
Giveaway."  The  Marshall  Project,  3  December  ,  2014,  accessed  November  17,  2016, 
https://www.themarshallproiect.org/2014/12/03/the-pentagon-finallv-details-its-weapons-for-cops- 

giveaway#.  A2XwBHEwA. 

44  Kevin  Sullivan,  Tom  Jackman  and  Brian  Fung,  "Dallas  Police  Used  a  Robot  to  kill.  What  Does  that  Mean  for  the 
Future  of  Police  Robots?"  The  Washington  Post.  21  July,  2016,  accessed  November  17,  2016, 
https://www.washingtonpost.com/national/dallas-police-used-a-robot-to-kill-what-does-that-mean-for-the-future-of- 

police-robots/2016/07/20/32eel  14e-4a84-l  Ie6-bdb9-701687974517  storv.html. 

45  W.J.  Hennigan  and  Brian  Bennett,  "Dallas  Police  Used  a  Robot  to  Kill  a  Gunman,  a  New  Tactic  that  Raises 
Ethical  Questions."  Los  Angeles  Times,  8  July,  2016,  accessed  November  17,  2016, 
http://www.latimes.com/nation/la-na-dallas-robot-20160708-snap-storv.html. 
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to  the  border  patrol  using  robots  to  run  drug  tunnels  found  beneath  the  border,  “if  anyone  is 
going  to  get  hurt,  it  better  be  that  robot.”47  Even  departments  of  wildlife,  parks  and  conservation 
share  the  same  concerns.  Many  of  those  agencies  respond  far  away  from  any  kind  of  backup,  in 
rural  and  remote  areas,  where  many  of  the  suspects  they  encounter  are  already  armed  with  high- 
powered  hunting  rifles.48  This  is  one  of  many  scenarios  where  former  military  specific 
technologies  are  beginning  to  replace  humans  to  reduce  the  risks  to  personnel.  The  future  use  of 
such  technology  also  seeks  to  reduce  risk  to  citizens  as  well 

While  police  and  border  agents  have  a  very  different  charge  to  protect  human  life  than 
does  the  military,  the  transfer  of  military  equipment  and  advancing  technology  is  providing  new 
opportunities  to  conduct  their  jobs  in  a  manner  safer  to  both  officers  and  citizens.49  Even  those 
who  are  most  opposed  to  LAS  see  similar  competitive  advantages  in  law  enforcement  as  those 
that  exist  for  military  personnel  The  greater  speed  of  processing  offered  by  LAS  could  easily  be 
applied  to  hostage  situations,  suicide  bomber  scenarios  and  relieve  humans  of  the  dull,  dangerous 
and  dirty  police  work.50  If  allowed,  improvements  in  facial  recognition  and  processing  speeds  for 
targeting  could  provide  an  ability  for  a  LAS  to  take  a  split  second  shot  when  humans  might  be 
too  slow  to  react.51  Other  potential  uses  include  crowd  control,  perimeter  protection  or  targeting 
a  certain  type  of  perpetrators  such  as  poachers  and  prison  rioters  or  escapees.52  The  trend  is 
already  moving  toward  autonomous  police  forces.53  South  Korea  has  invested  $44M  in 


47  Fernanda  Santos,  "Border's  New  Sentinels  Are  Robots,  Penetrating  DeepestDrug  Routes."  The  New  York  Times. 
22  February,  2014,  accessed  February  27,  2017,  https://www.nvtimes.com/2014/Q2/23/us/borders-new-sentinels- 
are-robots-penetrating-deepest-drug-routes.html?  r=0 

48  Musgrave,  “Pentagon  finally  details” 

49  Henningan  and  Bennett,  “Police  Used  a  Robot  to  Kill  a  Gunman” 

50  Christof  Heyns,  "Human  Rights  and  the  Use  of  Autonomous  Weapons  Systems  (AWS)  During  Domestic  Law 
Enforcement."  Human  Rights  Quarterly  38,  no.  2  (2016):  350-78,  doi:10.1353/hrq. 2016.0034. 

51  Ibid. 

52  Ibid.,  359 

53  Garcia.  “Why  the  US  Should  Lead  the  Ban,”  58. 
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developing  a  robot  police  force,  Russia  has  a  robot  that  polices  public  spaces  and  records 
offenses  with  a  camera,  and  the  UK  and  US  are  investing  in  micro  drones  for  surveillance.54 

Along  the  border,  DHS  sees  the  future  as  something  less  like  a  physical  wall  and  more 
like  the  next  evolution  of  the  Vietnam  era  McNamara  line.  Since  the  70’s,  they  have  been 
leveraging  that  technology  and  constantly  modernizing  to  create  a  “virtual  wall”  Their  vision  of 
the  future  includes  a  fused  network  of  sensors,  surveillance  towers,  drones,  blimps  and  other 
advanced  technological  equipment  to  provide  situational  awareness  and  response  to  the  US- 
Mexico  border.55  While  the  results  and  technology  have  come  along  way  since  the  Vietnam 
conflict,  the  border  surveillance  effort  is  still  manpower  intensive,  so  organizations  are 
continuing  to  enhance  their  networks  in  order  to  become  more  efficient  through  autonomy. 

In  2016,  DHS  issued  a  solicitation  for  innovative  ways  to  use  unmanned  vehicles.  While 
they  currently  use  a  mix  of  unmanned  maritime  vessels  and  aircraft,  they  are  expensive  to 
operate  on  a  budget  significantly  smaller  than  the  DoD.  Two  of  their  three  focus  areas  for 
funding  include  a  need  for  autonomous  technology.  The  first  area,  Sensors,  seeks  to  detect, 
identify  and  track  multiple  targets  autonomously.  The  second,  Platform  Security,  seeks  to  lay  in 
defensive  capabilities  against  unauthorized  actors  seeking  to  disrupt  or  disable  their  UAVs.56 
Israel’s  Viper  and  Gardium  systems  are  already  in  use  to  provide  similar  capabilities  in  their 
cities  without  the  need  for  a  pilot. 57  As  these  technologies  become  more  and  more  capable  of 
defensive  and  non-lethal  offensive  actions  it  is  becoming  clearer  that  LAS  may  be  available  for 


54  Ibid. 

55  Matt  Novak,  "US  Border  Patrol  Doesn't  Want  a  Wall-They  Want  Drones  and  Sensors."  Gizinodo,  28  April,  2016, 
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accessed  February  27,  2017,  http://www.federaltimes.com/story/govemment/dhs/2016/07/18/cbp-suas- 
call/87259176/. 
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domestic  law  enforcement  in  the  near  future  as  welL  LAS  use  in  law  enforcement  should  be 
scrutinized  even  harder  than  its  use  in  the  military  for  obvious  concerns  with  the  impacts  to  the 
rights  of  everyday  citizens.  Unfortunately,  their  development  has  not  attracted  the  same  kind  of 
attention  as  their  use  in  the  military.58  There  is  a  booming  industry  for  making  unmanned 
weapons  for  law  enforcement  and  despite  being  branded  as  “less  lethal,”  many  can  also  be  used 
in  a  different  mode  to  carry  out  deadly  force.59  Once  proven  in  the  domestic  sector,  the  ease  of 
conversion  of  this  technology  from  valid,  domestic  and  non-lethal  needs  to  lethal  functions  is  the 
second  main  driver  that  will  pave  the  way  for  their  acceptance  and  use  in  the  military. 

The  Slippery  Slope  of  Dual  Use  and  Selectable  Lethality 

There  are  seemingly  endless  applications  for  autonomous  technology  in  the  civilian 
world.  Search  and  rescue,  natural  disaster  infrastructure  assessments,  surveillance,  weather 
pattern  monitoring,  oceanographic  mapping,  driverless  cars,  even  the  Roomba  wondering  around 
our  homes  is  relying  on  a  kind  of  autonomous  technology.  These  applications  are  not  free  of 
ethical  concerns,  but  typically  they  are  more  closely  tied  to  personal  safety  and  privacy  rights.60 
However,  it  is  becoming  increasingly  more  difficult  to  separate  the  technologies  used  to  create 
“helpful”  autonomous  systems  from  those  designed  specifically  for  military  application. 

Developments  in  the  labs  are  advancing  similarly  to  the  swarm  tactics  being  used  by 
robots  in  the  field  to  solve  complicated  problems.61  There  is  no  central  control.  The  development 
of  the  technology  is  distributed  among  a  network  of  military  and  civilian  laboratories,  university 
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and  corporate  research  centers,  private  individuals  and  state-level  governments.  LAS  design 
incorporates  a  wide  range  of  technologies  whose  development  are  dependent  on  advances  from 
each  other  -  biotechnology,  robotics,  artificial  intelligence,  sensors  and  processors,  and 
information  systems  technologies.  They  rely  on  the  integration  of  material  and  sub-systems  that 
are,  in  many  cases,  commercial-off-the-shelf  products.  The  convergence  of  these  technologies 
has  accelerated  the  pace  at  which  progress  in  one  area  affects  progtess  in  another.62  Already, 
these  advances  in  multiple  technologies  are  moving  so  fast  that  regulations  on  their  use  are 
beginning  to  lag.63  From  Uber  replacing  a  highly  regulated  taxi  service  to  self- driving  cars,  the 
technology  is  proving  ready  for  use  well  before  local  or  state-level  communities  across  the  globe 
are  prepared  to  administrate  them64 

The  dual  use  appeal  has  presented  a  much  more  challenging  problem  than  has  been 
associated  with  potential  breakthrough  military  technologies  in  the  past.  In  contrast,  the 
development  of  nuclear  technology  was  not  only  more  centralized  in  development,  but  could  be 
connected  to  a  centralized  government  and  potential  users  to  aid  in  its  regulation.65  Other 
military  technologies,  such  as  blinding  lasers,  chemical  weapons,  and  biological  weapons  clearly 
did  not  have  an  intended  use  other  than  military  applications.  Even  with  centrally  controlled 
technologies  such  as  these,  regulation  was  still  no  easy  task.  By  the  time  the  US  atomic  bomb 
had  become  a  reality,  researchers  and  scientists  were  virtually  excluded  from  any  policy 
decisions  on  its  use.66  In  today’s  marketplace,  corporations  and  private  developers  continue  to 
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blur  the  line  as  they  find  more  and  more  civil  uses  outside  the  military  domain  for  robotics  and 


autonomous  technology.  As  these  products  are  developed,  fielded,  and  refined  to  overcome 
technological  hurdles  through  their  use  in  commercial  sectors,  the  temptation  and  ease  to 
weaponize  them  grows. 

Unmanned  underwater  vehicles  (UUVs)  serve  as  a  prime  example  of  this  phenomenon. 
UUVs  developed  for  oceanographic  mapping  turned  to  autonomous  operation  to  overcome  the 
limitations  with  radio  signals  traveling  underwater. 67  Once  proven  in  the  field,  the  technological 
gap  and  subsequent  approval  of  their  use  in  military  applications  such  as  mine  clearance,  anti¬ 
submarine  warfare  or  other  subsurface  attack  missions  grows  significantly  narrower.  Similarly, 
in  a  search  and  rescue  capacity,  microelectromechanical  systems  or  swarming  robots  need  to 
rapidly  coordinate  and  cooperate  in  real  time  without  direct  human  control.68  This  same 
technology,  once  proved  in  concept  in  the  civilian  world,  could  provide  significant  military 
capability  once  weaponized  with  relatively  little  additional  cost.  While  many  have  strong  moral 
aversions  to  LAS,  they  often  fail  to  see  the  similarities  in  autonomous  systems  intended  for 
civilian  use,  such  as  driverless  cars.  The  driverless  car  industry  is  addressing  similar  challenging 
moral  car  crash  scenarios  such  as  choosing  between  striking  a  child  vs.  an  elderly  pedestrian,  one 
person  vs.  a  crowd,  or  saving  the  driver  vs.  a  pedestrian.  Despite  utilizing  the  same  type  of 
technology  to  make  similar  morally  based  choices,  these  systems  are  mainly  seen  as  promising 
technologies  which  will  ultimately  serve  a  great  good  in  the  world.69 

Similar  to  non- lethal  and  dual  use  systems,  autonomous  systems  used  purely  for 
defensive  purposes  have  managed  to  avoid  the  greater  ethical  debates  concerning  their  use.  They 
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have  done  this  through  ensuring  the  systems  have  a  human  in  the  loop  for  either  the  direct 
command  to  fire  or  when  they  select  an  autonomous  mode  when  the  decision- to- fire  time  is 
prohibitive  for  a  human  to  ensure  protection  of  personnel  (such  in  missile  defense).  These 
systems  have  been  called  semi- autonomous,  automatic,  or  precursor  systems  and  are  already 
widely  in  use.70  However,  this  adjustable  autonomy  only  disguises  the  tme  autonomous 
capability  of  the  systems  which  could  easily  be  applied  to  other  platforms  in  a  more  offensive 
capacity.71  In  these  scenarios,  humans  are  increasingly  becoming  the  weakest  link  as  processing 
speeds  of  computers  outpace  human  decision-making  speed.  This  becomes  particularly 
problematic  when  multiple  systems  are  controlled  by  a  single  individual.  To  achieve  their  fullest 
potential,  these  systems  must  be  designed  from  the  beginning  to  function  autonomously,  free  of  a 
human  in  the  loop.  The  option  for  human  control  will  eventually  become  an  illusion  as  the  speed 
and  the  nature  of  weapons  designed  to  counteract  these  purely  defensive  systems  continues  to 
evolve. 

The  very  existence  of  such  autonomous  anti-missile  defensive  systems  such  as  Aegis, 

Iron  Dome  and  Patriot  are  driving  development  of  technologies  to  defeat  them  which  are  also 
autonomous  and  far  from  non-lethal.  In  these  defensive  systems,  a  sophisticated  radar  and 
sensors  which  can  track  multiple  inbound  threats  (or  targets)  are  integrated  into  weapons 
systems.  Without  direct  human  control,  these  systems  are  able  to  autonomously  launch  missiles 
or  fire  guns  to  neutralize  identified  threats.  In  one  such  future  scenario  designed  to  defeat  such 
systems,  a  disarming  attack  would  be  carried  out  by  small  swarming  offensive  weapons  which 
would  overwhelm  the  defensive  system’s  ability  to  track  and  target  all  the  potential  threats.72 
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This,  in  turn,  is  pushing  these  defensive  systems  away  from  their  stationary  positions,  expanding 
the  parameters  of  operation  beyond  their  current  tight  controls,  and  removing  more  and  more 
human  control  and  supervision.73  These  represent  many  of  the  same  qualities  which  would  also 
be  appealing  in  a  LAS.  Hie  cycle  of  development  for  LAS  is  already  starting  to  take  the  shape  of 
an  arms  race,  but  the  potential  for  dual  use  and  non-lethal  applications  has  many  urging  the 
technology  to  move  forward. 

In  the  April  2016  United  Nations  meeting  of  the  Convention  on  Certain  Conventional 
Weapons,  despite  a  growing  concern  over  LAS,  some  nations  felt  that  it  was  too  early  in  the 
development  of  autonomous  systems  to  make  a  decision  as  to  their  future.74  Advocates  feel  it  is 
important  to  test  the  limits  of  autonomous  capabilities  before  a  policy  can  be  developed  by  any 
governing  body  and  that  a  ban  on  weaponized  systems  could  prevent  the  development  of 
potentially  lifesaving  technologies.75  Irrational  fears  of  terminators  with  machine  guns  shouldn’t 
interfere  with  systems  such  as  missile  defense  which  could  ultimately  reduce  civilian 
casualties.76  Canada,  in  particular,  felt  that  it  was  too  difficult  to  separate  the  “good”  applications 
from  the  “bad”  and  an  outright  ban  would  be  detrimental  to  the  development  of  the  dual  use 
technology.77  In  areas  where  robotics  are  being  used  for  missions  like  land  mine  removal,  a 
highly  reflated  and  high  cost  advanced  system  loses  its  third  world  appeal.78  However,  these 
deliberations  may  be  end  up  being  merely  another  version  of  humans  justifying  the  things  we 
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want  to  do  by  attempting  to  control  the  context  of  their  use.  Unfortunately  with  LAS,  once  we 
lose  control  of  the  context,  there  may  be  no  going  back. 

The  Illusion  of  Control  and  Why  Context  Matters 

The  DoD  Unmanned  Systems  Integrated  Roadmap  recognizes  that  the  “current  state  of 
autonomous  systems  require  highly  structured  and  predictable  environments.”79  Going  forward, 
research  aims  to  either  better  define  and  control  the  environment  of  the  system  or  to  make  a 
machine  that  is  more  capable  of  interacting  with  the  environment  in  which  it  operates.  The 
former  is  much  easier  to  accomplish.  Current  autonomous  weapons  systems  are  only  deployed  in 
a  limited  context  into  environments  where  the  risk  to  human  life  is  small  and  the  benefits,  such 
as  protection  of  troops,  outweighs  the  risk. 80  This  is  mainly  seen  in  defensive  systems  that  are 
onboard  ships  or  protecting  troop  locations  in  contingency  environments. 

Any  deviations  away  from  an  environment  which  might  limit  a  commander’s  ability  to 
adhere  to  DoD’ s  “appropriate  levels  of  human  judgment  over  the  use  of  force”  policy  or  that  may 
require  unusual  uses  of  autonomous  technology  must  be  approved  by  a  senior-ranking  defense 
official81  These  lower  risk  environments  are  used  because  it  is  easier  to  limit  the  scope  of 
autonomous  operations  to  a  certain  mission  or  domain  with  low  civilian  populations  (i.e. 
underwater)  as  opposed  to  a  highly  complex  setting  such  as  an  urban  environment.  Current 
systems  perform  much  better  at  these  well-defined  problems  because  the  technology  is  not  yet 
capable  of  tasks  such  as  differentiating  between  combatant  and  noncombatant.  Thus  far, 
autonomous  technologies  have  seemingly  been  kept  “in  a  box”  to  control  their  effects  much 
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better  than  technologies  such  as  landmines  and  chemical  weapons.  Our  inability  to  control  the 
effects  of  the  latter  drove  the  international  community  toward  widely  accepted  treaties  which 
banned  their  use.82  However,  the  technological  ability  to  overcome  the  current  limitations  of 
LAS  is  approaching  very  rapidly.  As  these  incremental  advances  in  capability  are  achieved, 
control  of  the  system  is  evolving  in  a  parallel  track.  We  are  slowly  (or  perhaps  not  so  slowly) 
moving  from  full  command,  to  partial  command  aka  “human  in  the  loop,”  to  mere  oversight  of 
autonomous  systems.83  When  this  transition  is  complete,  the  issues  presented  by  a  machine  that 
is  more  capable  of  interacting  with  a  non- controlled  environment  must  be  addressed. 

Many,  such  as  Georgia  Tech’s  Ron  Arkin,  feel  that  autonomous  systems  can  be 
programmed  with  a  sort  of  “ethical  governor.”84  Once  the  technological  hurdles  such  as  facial 
recognition,  target  discrimination,  and  target  delineation,  among  others,  are  addressed,  he  feels 
that  a  LAS  could  be  programmed  based  on  any  set  of  current  governing  standards  in  the  conduct 
of  warfare.  These  might  include  international  laws,  specific  rules  of  engagement,  or  even  broader 
ethical  principles. 85  Once  these  systems  could  incrementally  be  proven  to  be  as  ethical  and 
effective  as  a  human  conducting  the  same  mission,  there  would  be  no  reason  why  they  shouldn’t 
be  employed.  Even  the  Pentagon  has  stated  that  when  the  technology  is  reliable  enough,  it  would 
be  open  to  the  possibility  of  a  LAS  being  able  to  fire  without  direct  human  permission.86 
Developers  are  not  trying  to  create  LAS  that  would  be  immune  from  ethical  mistakes,  merely 
ones  that  could  outperform  a  human  on  the  battlefield  given  the  same  situation  with  respect  to 
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international  law.87  While  meeting  these  ethical  standards  may  ultimately  lead  to  the  eventual 
acceptance  of  LAS  on  the  battlefield  via  public  opinion,  the  context  and  the  nature  of  future 
conflicts  may  have  more  influence  on  the  timeline  of  their  employment. 

Context  always  matters  and  the  development  timeline  and  eventual  deployment  of  LAS 
will  be  no  exception  to  this  rule.  Until  autonomous  systems  could  prove  to  be  operationally  and 
cost  effective  their  development  languished.  Once  their  use  was  proven  to  be  militarily  beneficial 
in  the  wars  in  Afghanistan  and  Iraq  (and  the  greater  war  on  terror),  their  development  and 
employment  has  exploded.  There  is  public  concern  over  the  use  of  LAS  just  as  there  was  for 
many  other  revolutionary  military  technologies  over  history.  Crossbows,  catapults,  and 
submarines  were  all  once  considered  inhumane  weapons  which  had  their  role  limited  in  combat. 
But  over  time  the  context  of  wars  changed  and  so  did  public  opinion  on  their  use.  Social 
conditioning  is  used  in  many  fields  to  slowly  overcome  concerns  of  the  general  population.88 
The  self-driving  car  is  being  tested  and  unveiled  methodically  and  incrementally  to  overcome 
public  fears  and  deep  rooted  social  norms.  The  environment  is  being  limited  and  controlled  to 
prevent  accidents  that  might  erode  public  confidence  in  the  cars  and  setback  their  eventual 
widespread  use  by  years  or  even  decades.  The  same  is  happening  with  LAS.  The  environments 
are  being  controlled  until  steady  technological  gains  allow  them  to  slowly  expand  in  capabilities, 
mission  sets  and  most  importantly,  public  support  and  trust.  However,  the  context  of  a  state’s 
situation  in  the  world  can  change  quickly  and  eliminate  the  need  for  a  slow  and  steady 
acceptance. 
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Opponents  of  the  introduction  of  LAS  frequently  point  to  the  Martens  Clause  and  the 
Law  of  Armed  Conflict.  In  the  absence  of  definitive  international  law  when  fielding  a  new 
weapons  system,  the  Martens  Clause  stresses  the  importance  of  customary  norms,  the  principles 
of  humanity,  and  the  dictates  of  the  public  conscience  in  the  conduct  of  warfare.89  The  public 
conscience  aspect  of  the  clause  is  frequently  cited  to  prevent  the  introduction  of  potentially 
disruptive  technologies  onto  the  battlefield,  such  as  LAS.  Public  conscience  has  proven  to  be 
fickle  time  and  time  again  and  context  plays  an  important  role  in  determining  which  side  of  the 
argument  the  public  rests.  Several  recent  research  efforts  have  attempted  to  highlight  this  fact. 

Most  surveys  in  regards  to  LAS  are  taken  in  a  vacuum  and  offer  a  simple  binary  choice 
without  context  -  “do  you  support  the  use  of  lethal  autonomous  systems.”  In  this  narrow  context, 
the  majority  of  those  polled  seem  to  oppose  their  introduction  and  cite  a  lack  of  trust  or  moral 
issues  with  outsourcing  killing  to  robots  asprimary  reasons  for  opposition.90  As  part  of  a 
YouGov  Omnibus  survey,  one  thousand  Americans  were  polled  on  the  employment  of  LAS,  but 
were  then  allowed  to  provide  open-ended  comments  to  explain  their  answer.  Out  of  this,  two  key 
important  contextual  patterns  became  evident.  First,  family  members  of  military  veterans  (not 
necessarily  the  vets  themselves)  tended  to  support  the  deployment  of  LAS  more.  Second,  both 
sides  valued  saving  human  lives  and  were  particularly  supportive  of  protecting  “our  troops”  from 
harm.91  While  the  basic  oppositions  may  be  in  line  with  the  premise  of  “meaningful  human 
control,”  the  comments  begin  to  underscore  that  opposition  is  contextual  and  could  be  swayed. 
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Michael  Horowitz  has  followed  up  this  YouGov  survey  in  an  attempt  to  determine  which 
factors  might  be  key  to  swaying  public  opinion  on  the  employment  of  LAS.92  While  overall 
public  knowledge  about  LAS  was  deemed  to  be  low,  there  was  also  significant  bias  of  opinion 
conjured  up  from  Hollywood  images  of  LAS  portrayed  in  movies  like  the  Terminator.  His 
survey  provided  a  series  of  if/then  options  in  regards  to  employment  of  LAS  instead  of  a 
traditional  binary  choice  of  support  vs.  not  support.  Three  factors  were  highlighted  that  would 
significantly  increase  support  for  LAS  -  if  they  protect  US  forces,  if  the  military  necessity 
increases,  or  if  other  countries  develop  them  first.  Opinion  shifted  from  only  38%  support  to 
61%  support  if  LAS  were  proven  to  be  more  effective  at  protecting  US  forces  and  up  to  50%  if 
proven  to  be  more  militarily  effective  in  attacking  the  enemy.  In  a  second  survey,  opposition  fell 
from  56%  to  only  38%  (68%  change)  if  another  country  developed  them  first.93  While  the  survey 
does  indicate  there  is  still  some  hesitation  to  support  their  employment  even  if  LAS  prove  to  be 
more  effective,  it  more  importantly  highlights  that  public  opinion  is  highly  malleable  on  the 
subject  depending  on  the  context.  As  these  are  hypothetical  scenarios  in  a  laboratory 
environment,  it  could  be  assumed  to  have  an  even  more  powerful  effect  on  public  opinion  if 
these  scenarios  played  out  in  the  real  world  with  significant  media  exposure  and  higher  stakes. 

One  informal  survey  that  might  be  the  most  telling  was  taken  at  the  2016  World 
Economic  Forum.  National  level  leaders  and  politicians  were  asked  two  questions  pertaining  to 
their  nation  suddenly  being  at  war.  The  first  asked  if  they  would  rather  be  defended  by  their  sons 
and  daughters  or  a  LAS.  The  second  asked  if  they  would  rather  be  invaded  by  sons  and  daughters 
of  their  enemy  or  that  same  LAS.  The  results  were  an  astounding  88%  that  preferred  to  be 
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defended  by  LAS  and  the  inverse  (12%)  which  preferred  to  face  LAS.94  While  these  numbers  are 
somewhat  lower  when  extended  to  larger  survey  population  it  does  highlight  the  ongoing  trend 
toward  casualty  avoidance  and  a  growing  confidence  in  technology  to  take  our  own  forces 
further  away  from  the  fight.  If  public  opinion  is  the  driver  of  the  future  employment  of  LAS, 
changes  in  context  throughout  history  have  shown  that  public  opinion  usually  lies  on  the  side  of 
self-interest  and  self-preservation.95  These  changes  in  context  also  fuel  emotions  and  irrational 
fears,  which  only  aid  in  the  acceleration  to  develop  and  field  LAS. 

Two  major  fears  are  driving  the  rapid  development  of  LAS,  despite  calls  to  slow  down 
and  re-examine  the  ethical  implications.  First,  there  is  significant  fear  over  current  systems  being 
stolen  by  the  enemy,  or  worse,  turned  against  its  own  forces  through  unsecure  networks  and 
datalinks.  There  is  also  a  growing  trend  in  electronic  warfare  that  will  require  systems  to  operate 
in  a  denied  environment,  where  communications  back  to  a  control  station  may  not  be  possible. 

To  be  successful  the  system  will  be  required  to  execute  missions  without  direct  human  control. 
Similar  to  underwater  environments  where  radio  signals  are  unable  to  reach  the  depth  and 
distances  required,  autonomy  provides  a  practical  solution.  A  self-contained  system  with  a 
preprogrammed  mission  or  decision  and  execution  matrix  that  isn’t  reliant  on  human  commands 
can  continue  to  work  even  in  the  era  of  quantum  computing,  where  no  current  encryption  will  be 
safe.96 

Second,  and  alluded  to  in  the  Horowitz  survey,  is  the  fear  that  if  “we  don’t  do  it,  someone 
else  will  get  them  first.”  This  has  driven  trends  in  warfare  and  arms  races  for  decades.  Hie  US 
nuclear  program  is  a  prime  example.  When  the  context  ofWWII  placed  the  US  in  front  of  an 
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existential  threat  and  the  prospects  for  tens  of  thousands  of  deaths  during  an  invasion  of 
mainland  Japan,  public  opinion  was  decidedly  swayed  and  opposition  to  having  used  the 
destmctive  power  of  the  atomic  bomb  were  significantly  lowered.  The  same  may  prove  tme  for 
LAS.  Deputy  Secretary  of  Defense  Bob  Work  has  stated  that  the  United  States  will  not  delegate 
lethal  decision-making  authority  to  a  machine,  but  followed  up  with  ‘but  self-restraint  might  be 
unsustainable  if  an  authoritarian  rival  acts  differently.”97  While  we  attempt  to  control  the  gradual 
employment  of  LAS  until  it  proves  to  be  reliable  and  effective,  he  recalls  the  old  adage  that  in 
war,  the  enemy  has  a  vote. 


Counter  Arguments 

Some  critics  of  LAS  do  not  see  their  employment  and  future  use  as  technologically 
inevitable.  Many  are  also  taking  active  roles  to  ensure  this  vision  of  the  future  doesn’t  happen 
and  are  calling  for  a  pre-emptive  ban  on  “killer  robots.”  Major  donors,  such  as  Elon  Musk,  are 
counted  among  those  who  are  working  to  “save  humanity”  from  the  dangers  of  artificial 
intelligence  and  LAS.  In  2015,  Musk  announced  $1  billion  in  funding  for  anew  non-profit  effort, 
OpenAI,  dedicated  to  deploying  new  technologies  only  in  ways  that  are  safe  for  humanity.98 
Prior  to  that,  14,000  researchers  penned  an  open  letter  through  the  Future  of  Life  Institute 
demanding  world  leaders  ban  LAS.99  Most  recently,  Human  Rights  Watch  has  championed  the 
“Campaign  to  Stop  Killer  Robots”  effort  to  push  the  UN  CCW  to  ban  such  technologies.  The 
ethical  reasons  discussed  in  this  paper,  fear  of  a  “killer  robot  arms  race,”  and  the  moral  issues 
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associated  with  a  machine  taking  a  human  life  without  a  “man  in  the  loop”  are  all  frequently 
cited  as  reasons  for  the  call  for  a  preemptive  ban.  Within  these  groups,  there  is  a  collective  fear 
that  if  we  don’t  get  a  handle  on  this  technology  now,  it  will  be  too  late  to  do  anything  about  it. 
Although  many  countries  have  policies  committed  to  responsible  development  of  LAS,  it  does 
little  to  dissuade  the  fear  that  “good  intentions  never  won  an  arms  race.”100  The  groups  pushing 
for  a  ban  feel  a  more  formal  mechanism  for  self-restraint  on  development  and  employment  of 
LAS  is  needed.  While  these  efforts  may  temporarily  slow  down  the  eventual  deployment  of 
LAS,  an  outright  ban  has  little  chance  to  succeed  for  many  reasons. 

There  are  several  types  of  weapons  that  have  been  subject  to  some  kind  of  international 
control  regime  -  anti-personnel  mines,  chemical  weapons,  blinding  lasers.  Those  weapons,  unlike 
LAS,  were  all  nearing  a  limited  military  value  even  bordering  on  becoming  obsolete  when  they 
were  banned. 101  This  is  certainly  not  the  case  with  LAS.  Now,  and  for  the  foreseeable  future, 
LAS  are  not  seen  as  niche  weapons  and  will  have  significant  strategic  importance  at  nearly  every 
level  and  domain  in  warfare.  As  systems  become  more  and  more  advanced  and  environments 
become  more  contested,  LAS  are  seen  as  more  and  more  of  a  military  necessity.  Their  upsides 
are  exceedingly  high  to  militaries  concerned  with  dull,  dirty,  or  dangerous  missions  that  are  also 
under  pressure  to  reduce  budgets  affected  by  rising  personnel  costs.  As  we  continue  to  see 
images  from  Syria,  Iraq  and  other  parts  of  the  world  where  there  are  seemingly  inordinate 
amounts  of  innocent  civilians  being  killed  in  war,  a  technology  that  potentially  offers  better 
precision,  discrimination  and  less  human  suffering  is  not  likely  to  be  turned  off  from  the 
development  pipeline.  As  discussed  previously,  context  and  public  opinion  is  malleable  and 
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without  a  strong  push  from  civil  society  to  lay  down  a  no-go  red- line  for  LAS,  their  development 
will  continue.102 

Furthermore,  no  weapon  system  has  ever  been  preemptively  banned  prior  to  their 
development  and  use.  Visions  of  Hollywood  terminators  are  generating  fears  based  on  “what  if’ 
scenarios  as  opposed  to  the  “what  was”  scenarios  that  drove  bans  in  the  past.  It  can  be  argued 
that  even  efforts  such  as  nuclear  non-proliferation,  generally  considered  to  be  successful  would 
not  be  possible  had  the  weapon’s  devastating  power  not  been  unleashed  first.  Too  much  money 
has  been  invested  in  LAS  worldwide  to  collectively  throw  in  the  towel  because  of  “what  if’ 
scenarios.  The  momentum  is  already  heavily  moving  in  the  direction  of  LAS  with  no  tangible 
proof  that  the  bad  will  outweigh  the  good.  In  fiscal  year  2013,  the  US  spent  almost  $6  billion  on 
unmanned  systems  and  predicts  the  unmanned  aerial  vehicle  market  to  more  than  double  in  the 
next  ten  years  reaching  nearly  $90  billion. 103  Admittedly,  the  majority  of  that  growth  is  predicted 
in  the  civilian  markets,  not  the  military. 

Finally,  if  there  was  to  be  an  attempt  at  an  outright  ban,  there  is  no  consensus  over  what 
grounds  or  legal  framework  LAS  would  fall  under  for  governance  or  control  Some  feel  it  is 
most  appropriate  under  human  rights  law,  others  feel  humanitarian  law  (which  deals  mainly  with 
its  application  during  wartime).  Others  feel  that  both  discussions  are  moot  as  autonomous  only 
refers  to  a  control  method  and  not  the  weapon  itself. 1 04  The  possession  of  a  weapon  system  in 
itself  is  not  illegal  but  rather  how  it  is  employed  according  to  preexisting  and  accepted 
international  laws  of  warfare.  In  this  case,  a  LAS  may  only  be  deemed  to  be  illegal  if  it  would 
cause  more  unnecessary  suffering  or  noncombatant  losses  than  the  current  weapons  systems 
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employed  today.  As  contextual  surveys  show,  if  LAS  were  proven  to  be  as  efficient  or  better 
than  current  systems,  their  legality  may  not  be  a  significant  concern.  With  so  many  unknowns 
about  the  true  future  capabilities  of  LAS,  no  hard  evidence  of  disproportionate  effects  on  the 
battlefield,  many  differing  legal  considerations,  and  obvious  dual  use  commercial  interest  an 
outright  ban  does  not  seem  likely  or  practical  Particularly,  a  ban  on  LAS  through  the  notoriously 
slow  UN  CCW  process  seems  more  likely  to  join  the  ranks  of  other  “good  ideas  that  go  on  to  die 
a  slow  death.”  105 

Conclusion 

In  the  not  too  distant  future,  we  will  see  LAS  on  the  battlefield  either  working  with  our 
human  soldiers  or  totally  independently.  There  are  several  efforts  underway  to  limit  or  prevent 
their  eventual  use,  but  as  those  discussions  slowly  advance  the  development  of  LAS  technology 
is  moving  at  a  blinding  speed.  By  the  time  the  question  of  their  ethical  or  legal  use  in  the  military 
is  settled  on  the  international  stage,  it  will  be  too  late  to  change  the  path  and  momentum  of  LAS 
technology.  Some  future  analysis  on  why  the  efforts  to  ban  LAS  failed  will  likely  identify  three 
primary  reasons.  First,  the  same  technology  will  have  proved  itself  in  law  enforcement  and 
border  patrol  type  applications  with  relatively  little  scrutiny  as  compared  to  its  military  use. 
Secondly,  once  the  technologies  such  as  facial  recognition,  target  identification,  tracking  and 
targeting  have  been  proven  in  the  field  to  be  reliable  and  efficient,  the  conversion  for  lethal 
military  application  will  be  relatively  simple.  Finally,  the  context  of  LAS  use  will  be  tightly 
controlled  until  a  point  when  public  opinion  is  supportive  of  their  use  due  to  military  necessity, 
efficiency,  or  a  variety  of  other  factors.  Many  feel  that  will  be  the  point  of  no  return.  If  LAS  on 
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the  battlefield  is  a  technical  inevitability,  instead  of  relying  on  a  hope  that  they  can  be  banned 
outright,  we  should  work  to  establish  acceptable  parameters  of  their  use. 

Re  corame  ndations 

The  world-renowned  Stockton  Center  for  the  Study  of  International  Law  at  the  U.S.  Navy 
War  College  has  done  extensive  work  on  the  legal  aspects  of  LAS.  They  have  been  on  the 
forefront  of  many  issues  associated  with  the  introduction  of  LAS  on  to  the  battlefield  and  the 
recommendations  that  follow  are  based  on  some  of  their  initial  work.  To  ensure  that  when  LAS 
are  introduced  on  the  battlefield  they  are  used  in  a  way  that  satisfies  ethical  and  legal  concerns, 
the  following  initiatives  and  parameters  are  recommended.  All  should  be  perused  concurrently  as 
many  provide  overlap  and  are  mutually  supportive.  While  the  adaptation  of  these  standards  by  an 
internationally  recognized  global  organization  such  as  the  United  Nations  or  through  an 
international  treaty  would  provide  the  optimum  solution,  the  United  States  should  lead  the  way 
and  develop  policies  to  enact  these  recommendations  internally  first.  Recommendations  #2-5 
should  be  internally  implemented  relatively  quickly  to  aid  in  setting  the  roadmap  and 
requirements  for  future  development. 

1.  Develop  a  US-led  international  consensus  on  the  acceptable  uses  of  LAS.  The  use  of  UAVs 
in  Pakistan  and  Afghanistan  has  somewhat  eroded  the  US  credibility  in  the  ethical  debate 
about  the  employment  of  LAS.  Now  is  the  time  to  reassert  the  US  lead  in  the  responsible 
development  and  employment  of  LAS  in  the  future.  As  Singer  and  Rolf  write,  this  must  be 
done  sequentially  at  three  distinct  levels  -  through  our  current  partners  and  allies,  then  with 
regional  groups,  then  at  the  global  level  with  our  near  peer  competitors  such  as  China  and 
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Russia. 106  Most  importantly,  this  will  establish  that  the  international  community 
acknowledges  that  these  systems  will  become  reality  and  a  ban  is  not  practical  Once  this  has 
been  settled,  the  international  community  can  move  forward  with  placing  boundaries  on  how 
LAS  will  be  used.  Some  precedent  has  already  been  set  through  US  statements  at  the  CCW 
and  the  publication  of  DoDD  300.09. 

2.  Restrict  use  of  LAS  to  only  platform-level  targets.  This  capability  already  exists  in  some 
forms  today.  To  avoid  the  ethical  and  moral  challenges  of  targeting  individuals  on  the 
battlefield,  LAS  should  only  be  utilized  to  target  large  systems  such  as  enemy  aircraft,  naval 
forces,  radar  sites,  missile  systems,  etc.  While  challenges  remain  with  proper  target 
identification  and  selection,  characteristics  such  as  radar  signatures,  hull  profiles,  and 
ftiend/foe  protocols  are  much  more  discemable  between  military  systems  than  are  possible 
between  combatants  and  noncombatant  civilians.  There  would  be  no  distinction  between 
manned  and  unmanned  systems  during  wartime  as  their  nature  is  inherently  military 
regardless  of  how  the  system  is  operated.  However,  this  will  not  alleviate  legal  concerns  with 
proportionality,  which  will  also  need  to  be  sufficiently  met. 

3.  Require  every  LAS  to  be  identifiable  and  attributable  to  the  country  that  employs  them.  This 
is  no  different  than  the  flag  that  each  member  of  a  nation’s  armed  forces  wears  on  their 
distinct  uniform.  The  same  is  represented  in  the  tail  number  of  an  aircraft.  The  lack  of 
adherence  to  this  standard  is  one  of  the  reasons  terrorist  organizations  are  universally 
repudiated  by  the  international  community.  The  requirement  for  LAS  should  be  the  same. 
This  would  serve  to  address  the  concern  of  attribution  and  limit  the  chance  that  a  non- state 
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actor  could  release  a  system  “to  the  world”  without  fear  of  attribution  or  repercussion  from 
the  international  community.  Most  major  military  platforms  in  use  today  follow  the  same 
requirement  of  marking  and  identification. 

4.  Restrict  the  domains  where  a  LAS  might  be  employed.  Some  domains,  such  as  underwater  or 
air,  are  less  burdened  with  the  potential  for  unnecessary  civilian  casualties.  In  domains  where 
the  risk  would  be  low  and  would  see  more  system  vs.  system  level  clashes  (i,e.  submarine  or 
air  to  air),  LAS  could  be  employed  with  limited  restriction.  In  other  domains  or 
environments,  such  as  urban  combat,  a  LAS  would  be  much  more  problematic.  In  these 
cases,  where  risk  to  civilians  is  high,  it  would  be  more  appropriate  to  keep  a  human  in  the 
loop.  However,  LAS  could  be  employed  in  urban  areas  provided  recommendation  #2  was 
followed  and  they  were  restricted  to  military  platform  based  targets  only.  This  may,  however, 
increase  the  risk  of  enemy  violations  of  codified  laws  of  armed  conflict,  such  as  surface  to  air 
missiles  near  hospitals.  The  restriction  on  environment  may  happen  automatically  if 
sufficient  technology  doesn’t  exist  to  allow  a  LAS  to  act  with  appropriate  distinction  in  urban 
environments. 

5.  Restrict  the  use  of  LAS  to  only  conflicts  with  declared  state  on  state  hostilities.  The  same 
moral  calculus  forgoing  to  war  should  still  apply  regardless  of  the  technology  available. 

Fears  that  LAS  might  lower  the  barriers  required  to  go  to  war  may  be  mitigated  through  the 
requirement  to  formally  declare  war  or  recognize  the  beginning  of  hostilities  at  the 
international  level  This  would  go  ter  further  stigmatize  the  use  of  LAS  by  non- state  actors 
and  potentially  raise  the  threshold  before  LAS  could  be  “switched  on.”  If  this  level  of 
notification  is  not  achievable,  an  alternative  would  be  a  formal  notification  to  the  UN 
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Security  Council  of  LAS  employment  like  is  done  when  states  act  in  self-defense  under  Art. 
51. 

6.  Better  define  “defensive  systems.  ”  There  are  autonomous  systemsconducting  defensive 
missions  today  with  much  defined  parameters.  As  the  capability  of  autonomous  systems 
expands,  the  defensive  rings  of  protection  potentially  do  as  well.  How  far  out  will  this 
extend  on  a  ship  vs.  a  base  perimeter?  Will  they  be  required  to  be  stationary  or  can  they 
become  mobile  or  airborne?  Can  they  follow  a  squad  of  marines  on  the  ground?  At  what 
level  of  target  (system  vs.  human)  will  they  be  able  to  engage?  All  must  be  clearly  defined  as 
much  of  the  same  technology  will  be  utilized  in  purely  offensive  systems  as  well. 

7.  Internationally  ban  LAS  and  the  option  of  selectable  lethality  from  use  in  all  non-military 
applications.  This  would  serve  to  reinforce  recommendation  #5.  While  the  technology 
would  certainly  be  appealing  for  a  dictator  to  suppress  uprisings,  riots  or  to  conduct 
aggressive  crowd  control  this  cannot  be  tolerated  internationally.  In  those  cases,  even  the 
over  aggressive  use  of  non- lethal  systems  must  be  highly  scrutinized  as  well  Additionally,  it 
may  not  seem  like  a  likely  scenario  now,  but  there  are  some  compelling  arguments  to  give 
police,  federal  agents  and  even  commercial  security  organizations  this  capability  in  the 
future.  We  should  hold  the  domestic  application  of  this  technology  to  a  much  higher 
standard,  but  yet  systems  are  being  developed  today  to  readily  go  from  non-lethal 
applications  to  lethal  ones.  Because  of  uniquely  different  moral  and  ethical  calculations  and 
due  process  for  citizens,  these  systems  should  always  require  a  human  in  the  loop  and  never 
be  designed  specifically  for  lethal  operations.  This  recommendation  is  also  predicated  by  the 
ability  to  come  to  an  internationally  agreed  upon  definition  of  a  LAS  and  a  distinction  from 
purely  defensive  systems. 
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We  may  not  be  able  to  stop  the  eventual  employment  of  Lethal  Autonomous  Systems,  but 
with  all  these  recommendations  in  place,  we  may  be  able  to  set  the  conditions  where  they  are 
employed  in  the  most  ethical  moral  and  responsible  manner.  If  we  don’t  act  soon,  it  may  be  too 
late  to  have  a  say.  The  technology  itself  is  not  inherently  evil  but  how  we  choose  to  employ  (or 
not  employ)  it  will  determine  the  future  of  modem  conflict  and  maybe  even  how  we  define 
humanity  itself. 
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